Area-based breast percentage density estimation in mammograms using weight-adaptive multitask learning

Breast density, which is a measure of the relative amount of fibroglandular tissue within the breast area, is one of the most important breast cancer risk factors. Accurate segmentation of fibroglandular tissues and breast area is crucial for computing the breast density. Semiautomatic and fully automatic computer-aided design tools have been developed to estimate the percentage of breast density in mammograms. However, the available approaches are usually limited to specific mammogram views and are inadequate for complete delineation of the pectoral muscle. These tools also perform poorly in cases of data variability and often require an experienced radiologist to adjust the segmentation threshold for fibroglandular tissue within the breast area. This study proposes a new deep learning architecture that automatically estimates the area-based breast percentage density from mammograms using a weight-adaptive multitask learning approach. The proposed approach simultaneously segments the breast and dense tissues and further estimates the breast percentage density. We evaluate the performance of the proposed model in both segmentation and density estimation on an independent evaluation set of 7500 craniocaudal and mediolateral oblique-view mammograms from Kuopio University Hospital, Finland. The proposed multitask segmentation approach outperforms and achieves average relative improvements of 2.88% and 9.78% in terms of F-score compared to the multitask U-net and a fully convolutional neural network, respectively. The estimated breast density values using our approach strongly correlate with radiologists’ assessments with a Pearson’s correlation of \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$r = 0.90$$\end{document}r=0.90 (95% confidence interval [0.89, 0.91]). We conclude that our approach greatly improves the segmentation accuracy of the breast area and dense tissues; thus, it can play a vital role in accurately computing the breast density. Our density estimation model considerably reduces the time and effort needed to estimate density values from mammograms by radiologists and therefore, decreases inter- and intra-reader variability.

B Finding optimal hyperparameters using the Bayesian optimization search algorithm Table S1 presents different hyperparameters included in the Bayesian optimization search algorithm and their found optimal values. In our experiments, we used the multitask segmentation model trained with the Adam optimizer at an initial learning rate of le-3, the "reduced learning rate on plateau" as the learning rate scheduler, and the focal Tversky as the loss function to achieve superior segmentation performance in terms of IoU on the validation set of all the datasets. Table S1. Hyperparameter tuning using the Bayesian optimization search algorithm. The proposed multitask segmentation is fine-tuned using the validation set of all the datasets. The optimal hyperparameters are highlighted in the last column.

Hyperparameters
Search C Effect of batch size and normalization techniques on the multitask segmentation accuracy Figure S2 demonstrates the effect of various normalization techniques at different batch sizes on segmentation performance. We noticed that the normalization techniques degrade the segmentation accuracy with a small batch size of 1 or 2 and further generate the singularity caused by the nonlinear activation function (ReLU) in the convolutional layers of the segmentation models. The multitask segmentation model trained at a batch size of 4 with the combined weight standard and instance normalization (IN) techniques shows superior performance compared with different combinations of batch sizes and normalization techniques. With batch size increased to greater than 4, the combination of weight standard with IN shows consistent performance, while the performances of other normalization techniques degrade slightly. With the larger batch size of 16, the model freezes due to computational memory error (GPU memory). The weight standard technique combined with batch normalization (BN) improved the segmentation accuracy by average relative improvements of 6.17% and 5.88% in terms of IoU and F-score, respectively, compared to the model trained with only BN. We trained and evaluated the proposed and the baseline multitask segmentation models with the optimal hyperparameters with a batch size of 4 and the combined weight standard and IN normalization techniques.

D Visualizations of the breast area and the dense tissue segmentations predicted by the MTLSegNet
Here, we show a few examples from all datasets for the breast-area and dense-tissue segmentations predicted by MTLSegNet. We resized all the images to 256x256, and no further pre-processing techniques were applied. The qualitative visualizations are illustrated in Figure S3 and Figure S4 for the CC-and MLO-view mammograms, respectively, on each individual dataset. Figure S3. The predicted breast-area and dense-tissue segmentation's on the CC-view mammograms of the evaluation set for the INbreast, KUH, and mini-DDSM datasets. The red contour represents the predicted breast area, and the green pixels represent the predicted dense tissues. Figure S4. The predicted breast-area and dense-tissue segmentations on the MLO-view mammograms of the evaluation set for the INbreast, KUH, and mini-DDSM datasets. The red contour represents the predicted breast area, and the green pixels represent the predicted dense tissues.

5/7
G MTLSegNet more accurately segments the breast area and the dense tissues compared to the LIBRA Figure S7 and Figure S8 visually compare the breast-area and dense-tissue segmentations predicted by the LIBRA and MTLSegNet models for two examples from the KUH evaluation set. We rescaled the output of LIBRA and MTLSegNet to the input mammogram resolution. Figure S7 shows that LIBRA ineffectively segmented the blood vessels as dense tissues, resulting in dense-tissue over-segmentation in the MLO-view mammogram. Figure S7. An example showing that LIBRA segmented the blood vessels as fibroglandular tissues, while MTLSegNet successfully discriminated between the blood vessels and the dense tissues within the breast area. Red contours denote the predicted breast area.
The example in Figure S8 shows that MTLSegNet successfully excluded the pectoral muscle and other tissues from the breast-area segmentation. LIBRA often segments the pectoral and abdominal tissues as breast area, thus resulting in an overestimate of breast density by an average of 5% on the KUH evaluation set. Figure S8. An example demonstrating that LIBRA fails to exclude the pectoral and other tissues from the breast-area segmentation in the MLO-view images. Red contours denote the predicted breast area.